Repeatable semantic reef-mapping through photogrammetry and label-augmentation

In an endeavor to study natural systems at multiple spatial and taxonomic resolutions, there is an urgent need for automated, high-throughput frameworks that can handle plethora of information. The coalescence of remote-sensing, computer-vision, and deep-learning elicits a new era in ecological research. However, in complex systems, such as marine-benthic habitats, key ecological processes still remain enigmatic due to the lack of cross-scale automated approaches (mms to kms) for community structure analysis. We address this gap by working towards scalable and comprehensive photogrammetric surveys, tackling the profound challenges of full semantic segmentation and 3D grid definition. Full semantic segmentation (where every pixel is classified) is extremely labour-intensive and difficult to achieve using manual labeling. We propose using label-augmentation, i.e., propagation of sparse manual labels, to accelerate the task of full segmentation of photomosaics. Photomosaics are synthetic images generated from a projected point-of-view of a 3D model. In the lack of navigation sensors (e.g., a diver-held camera), it is difficult to repeatably determine the slope-angle of a 3D map. We show this is especially important in complex topographical settings, prevalent in coral-reefs. Specifically, we evaluate our approach on benthic habitats, in three different environments in the challenging underwater domain. Our approach for label-augmentation shows human-level accuracy in full segmentation of photomosaics using labeling as sparse as 0.1%, evaluated on several ecological measures. Moreover, we found that grid definition using a leveler improves the consistency in community-metrics obtained due to occlusions and topology (angle and distance between objects), and that we were able to standardise the 3D transformation with two percent error in size measurements. By significantly easing the annotation process for full segmentation and standardizing the 3D grid definition we present a semantic mapping methodology enabling change-detection, which is practical, swift, and cost-effective. Our workflow enables repeatable surveys without permanent markers and specialized mapping gear, useful for research and monitoring, and our code is available online. Additionally, we release the Benthos data-set, fully manually labeled photomosaics from three oceanic environments with over 4500 segmented objects useful for research in computer-vision and marine ecology

Use of commercial off-the-shelf digital cameras for scientific data acquisition and scene-specific color calibration

Author Posting. © Optical Society of America, 2014. This article is posted here by permission of Optical Society of America for personal use, not for redistribution. The definitive version was published in Journal of the Optical Society of America A: Optics, Image Science, and Vision 31 (2014): 312-321, doi:10.1364/JOSAA.31.000312.Commercial off-the-shelf digital cameras are inexpensive and easy-to-use instruments that can be used for quantitative scientific data acquisition if images are captured in raw format and processed so that they maintain a linear relationship with scene radiance. Here we describe the image-processing steps required for consistent data acquisition with color cameras. In addition, we present a method for scene-specific color calibration that increases the accuracy of color capture when a scene contains colors that are not well represented in the gamut of a standard color-calibration target. We demonstrate applications of the proposed methodology in the fields of biomedical engineering, artwork photography, perception science, marine biology, and underwater imaging.T. Treibitz is an Awardee of the Weizmann Institute of Science—National Postdoctoral Award Program for Advancing Women in Science and was supported by NSF grant ATM-0941760. D. Akkaynak, J. Allen, and R. Hanlon were supported by NSF grant 1129897 and ONR grants N0001406-1-0202 and N00014-10-1-0989 and U. Demirci by grants R01AI093282, R01AI081534, and NIH U54EB15408. J. Allen is grateful for support from a National Defense Science and Engineering Graduate Fellowship

Report on the Marine Imaging Workshop 2017

Marine optical imaging has become a major assessment tool in science, policy and public understanding of our seas and oceans. Methodology in this field is developing rapidly, including hardware, software and the ways of their application. The aim of the Marine Imaging Workshop (MIW) is to bring together academics, research scientists and engineers, as well as industrial partners to discuss these developments, along with applications, challenges and future directions. The first MIW was held in Southampton, UK in April 2014. The second MIW, held in Kiel, Germany, in 2017 involved more than 100 attendees, who shared the latest developments in marine imaging through a combination of traditional oral and poster presentations, interactive sessions and focused discussion sessions. This article summarises the topics addressed during the workshop, particularly the outcomes of these discussion sessions for future reference and to make the workshop results available to the open public

Perspectives in visual imaging for marine biology and ecology: from acquisition to understanding

Durden J, Schoening T, Althaus F, et al. Perspectives in Visual Imaging for Marine Biology and Ecology: From Acquisition to Understanding. In: Hughes RN, Hughes DJ, Smith IP, Dale AC, eds. Oceanography and Marine Biology: An Annual Review. 54. Boca Raton: CRC Press; 2016: 1-72

Adjustment and Calibration of Dome Port Camera Systems for Underwater Vision

Dome ports act as spherical windows in underwater housings through which a camera can observe objects in the water. As compared to flat glass interfaces, they do not limit the field of view, and they do not cause refraction of light observed by a pinhole camera positioned exactly in the center of the dome. Mechanically adjusting a real lens to this position is a challenging task, in particular for those integrated in deep sea housings. In this contribution a mechanical adjustment procedure based on straight line observations above and below water is proposed that allows for accurate alignments. Additionally, we show a chessboard-based method employing an underwater/above-water image pair to estimate potentially remaining offsets from the dome center to allow refraction correction in photogrammetric applications. Besides providing intuition about the severity of refraction in certain settings, we demonstrate the methods on real data for acrylic and glass domes in the water

Fish assemblages of three common artificial reef designs during early colonization

In this study, we compared the early fish colonization of three types of artificial reefs deployed in the coastal waters of Saba and St Eustatius in the Caribbean: reef balls®, layered cakes and piles of locally obtained basaltic rock. As an indicator of performance, three fish assemblage parameters (abundance, biomass, species richness) were measured using underwater visual censuses at 11 months post-deployment and 4 months after restoration from hurricane damage. All artificial reef plots showed higher values for fish abundance, biomass and species richness than control plots covered by bare sand, which shows that artificial reefs can locally enhance the fish assemblage. However, the effect differed among artificial reef plots. Fish abundance was 3.8 times higher on the layered cake plots compared to the reef ball plots, while fish biomass was 4.6 times higher. Rock pile plots had intermediate values. Species richness did not differ significantly among different artificial reef plots. Three-dimensional modelling revealed that layered cakes had a smaller gross volume, shelter volume and total surface area than reef balls. The availability of multiple small shelters in the layered cake design appeared to be more relevant than other physical parameters, as the layered cake plots had higher fish abundance than the reef balls plots. We concluded that on Saba and St. Eustatius, layered cake plots performed better than reef ball plots after one year of colonization. Rock pile plots, made of local volcanic rock, showed an intermediate performance, and were 4–10 times cheaper to construct. If observed differences are consistent with other locations and persist during further colonization, current efforts to deploy reef balls could better be allocated to deploy artificial reef structures with a higher shelter density